An Audio-Haptic Aesthetic Framework Influenced by Visual Theory

Sound is touch at a distance. The vibration of pressure waves in the air creates sounds that our ears hear, at close range, these pressure waves may also be felt as vibration. This audio-haptic relationship has potential for enriching interaction in human-computer interfaces. How can interface designers manipulate attention using audio-haptic media? We propose a theoretical perceptual framework for design of audio-haptic media, influenced by aesthetic frameworks in visual theory and audio design. The aesthetic issues of the multimodal interplay between audio and haptic modalities are presented, with discussion based on anecdotes from multimedia artists. We use the aesthetic theory to develop four design mechanisms for transition between audio and haptic channels: synchronization, temporal linearization, masking and synchresis. An example composition using these mechanisms, and the multisensory design intent, is discussed by the designers.